Image registration of intraoral images using adhesive objects

ABSTRACT

Data from an intraoral scan is received, the data comprising a plurality of intraoral images of a dental site. At least two intraoral images are identified that comprise a representation of at least a portion of a non-rigid object that was affixed to the dental site at a target area. Image registration is performed between the at least two intraoral images using the non-rigid object identified in the at least two intraoral images. A 3D model of the dental site is generated based on the image registration. A representation of the non-rigid object is subtracted from the 3D model based on the known properties of the non-rigid object. A surface of a portion of the dental site is interpolated where the non-rigid object was located based on a) data for other portions of the dental site and b) the surface of the base of the non-rigid object.

RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/165,504, filed Jan. 27, 2014, which is incorporated byreference herein.

TECHNICAL FIELD

Embodiments of the present invention relate to the field of intraoralscanning and, in particular, to a system and method for improving theresults of intraoral scanning in oral cavities that lack one or moreteeth.

BACKGROUND

In prosthodontic procedures designed to implant a dental prosthesis inthe oral cavity, the dental site at which the prosthesis is to beimplanted in many cases should be measured accurately and studiedcarefully, so that a prosthesis such as a crown, denture or bridge, forexample, can be properly designed and dimensioned to fit in place. Agood fit enables mechanical stresses to be properly transmitted betweenthe prosthesis and the jaw, and to prevent infection of the gums via theinterface between the prosthesis and the dental site, for example.

Some procedures also call for removable prosthetics to be fabricated toreplace one or more missing teeth, such as a partial or full denture, inwhich case the surface contours of the areas where the teeth are missingneed to be reproduced accurately so that the resulting prosthetic fitsover the edentulous region with even pressure on the soft tissues.

In some practices, the dental site is prepared by a dental practitioner,and a positive physical model of the dental site is constructed usingknown methods. Alternatively, the dental site may be scanned to provide3D data of the dental site. In either case, the virtual or real model ofthe dental site is sent to the dental lab, which manufactures theprosthesis based on the model. However, if the model is deficient orundefined in certain areas, or if the preparation was not optimallyconfigured for receiving the prosthesis, the design of the prosthesismay be less than optimal. For example, if the insertion path implied bythe preparation for a closely-fitting coping would result in theprosthesis colliding with adjacent teeth, the coping geometry has to bealtered to avoid the collision, which may result in the coping designbeing less optimal. Further, if the area of the preparation containing afinish line lacks definition, it may not be possible to properlydetermine the finish line and thus the lower edge of the coping may notbe properly designed. Indeed, in some circumstances, the model isrejected and the dental practitioner then re-scans the dental site, orreworks the preparation, so that a suitable prosthesis may be produced.

In orthodontic procedures it can be important to provide a model of oneor both jaws. Where such orthodontic procedures are designed virtually,a virtual model of the oral cavity is also beneficial. Such a virtualmodel may be obtained by scanning the oral cavity directly, or byproducing a physical model of the dentition, and then scanning the modelwith a suitable scanner.

Thus, in both prosthodontic and orthodontic procedures, obtaining athree-dimensional (3D) model of a dental site in the oral cavity is aninitial procedure that is performed. When the 3D model is a virtualmodel, the more complete and accurate the scans of the dental site are,the higher the quality of the virtual model, and thus the greater theability to design an optimal prosthesis or orthodontic treatmentappliance(s).

Scanning of the dental site is complicated by regions in which a patientis missing teeth, referred to as edentulous regions. For example, incases where two or more adjacent teeth are missing, there may be a largespan of soft tissue that needs to be scanned. Such regions can bedifficult to scan.

Some intraoral scanners are used in conjunction with a powder that isapplied to a dental region. The powder may include particles thatreflect light, with the goal of providing measurable points in thedental site. For such systems, these particles may be used to aid imageregistration when they operate as intended. However, the powder oftendoes not connect well to soft tissue, and in particular to wet softtissue. Additionally, the powder may become wet and/or wash away duringscanning, decreasing an accuracy of later image registration.Additionally, many patients do not like having the powder applied totheir teeth and in their mouth. Having to powder the teeth can havedrawbacks such as:

-   1. All areas have to be powdered and the thickness of the powder    layer is not homogeneous, which compromises accuracy (e.g., since    the surface is not scanned directly);-   2. If the scanner head touches the powder, it sticks to the optics    and introduces noise to the scan;-   3. The powder can be costly;-   4. Some people are allergic to the powder; and-   5. Color scanning of the teeth is not possible as it is all painted    in white.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1 illustrates a flow diagram for a method of performing intraoralscanning, in accordance with embodiments of the present invention.

FIG. 2 illustrates a flow diagram for a method of performing imageregistration of intraoral images, in accordance with embodiments of thepresent invention.

FIG. 3 illustrates one embodiment of a system for performing intraoralscanning and generating a virtual three dimensional model of a dentalsite.

FIG. 4A illustrates a portion of an example dental arch that is missingtwo teeth.

FIG. 4B illustrates the example dental arch of FIG. 4A with the additionof an adhesive object placed at the location of the missing teeth, inaccordance with embodiments of the present invention.

FIG. 5A illustrates a first intraoral image that includes a firstrepresentation of an adhesive object, in accordance with one embodimentof the present invention.

FIG. 5B illustrates a second intraoral image that includes a secondrepresentation of an adhesive object, in accordance with one embodimentof the present invention.

FIG. 5C illustrates a successful image registration of the firstintraoral image of FIG. 5A and the second intraoral image of FIG. 5B, inaccordance with one embodiment of the present invention.

FIG. 5D illustrates a first intraoral image that includes a firstrepresentation of a portion of an adhesive object, in accordance withone embodiment of the present invention.

FIG. 5E illustrates a second intraoral image that includes a secondrepresentation of a different portion of the adhesive object, inaccordance with one embodiment of the present invention.

FIG. 5F illustrates a successful image registration of the firstintraoral image of FIG. 5D and the second intraoral image of FIG. 5E, inaccordance with one embodiment of the present invention.

FIG. 5G illustrates a typodont, in accordance with one embodiment.

FIG. 5H illustrates a reconstruction of a typodont, in accordance withone embodiment.

FIG. 6A illustrates a portion of an edentulous arch with the addition ofa pattern of adhesive objects placed at various locations, in accordancewith embodiments of the present invention.

FIG. 6B illustrates a portion of an edentulous arch with the addition ofa pattern of adhesive objects placed at various locations, in accordancewith embodiments of the present invention.

FIG. 7 illustrates example adhesive objects, in accordance withembodiments of the present invention.

FIG. 8 illustrates example chains of interconnected adhesive objects, inaccordance with embodiments of the present invention.

FIG. 9 illustrates a block diagram of an example computing device, inaccordance with embodiments of the present invention.

DETAILED DESCRIPTION

Described herein is a method and apparatus for improving the quality ofintraoral scans taken of dental sites for patients missing some or allof their teeth. One or more adhesive objects may be placed at dentalsites in a patient's oral cavity where teeth are missing. The adhesiveobjects may have a body with a particular shape, may have an adhesive ona lower surface of the body, and may have a feature on an upper surfaceof the body. The shape of the body and/or the feature on the uppersurface (e.g., including a pattern and/or one or more colors) provides ageometrical or optical reference point for image registration of imagesgenerated by an intraoral scanner. In one embodiment, the body of theadhesive object is composed of a different material than the adhesive.In one embodiment, the adhesive object has a structure, shape, orpattern that does not substantially change during intraoral scanning(e.g., that does not wash away, such as may occur with powders). Theadhesive object may also be an ink that is stamped onto the dental sites(e.g., onto a gingival surface) with a particular shape or pattern. Thestamped ink may provide a reference point for image registration.

In one embodiment, a processing device receives intraoral images from anintraoral scan of a patient. The intraoral images may be discrete images(e.g., point-and-shoot images) or frames from an intraoral video (e.g.,a continuous scan). Some of the intraoral images may includerepresentations of an adhesive object (or multiple adhesive objects)affixed to a dental site (e.g., attached to the gingival tissue in adental arch) at a target area. The adhesive object may have knownproperties and provides a reference point for image registration. Theprocessing device performs image registration of the intraoral images.This may include performing image registration (i.e., image “stitching”)between at least two intraoral images (scans) that each include arepresentation of at least a portion of the adhesive object. If theproperties of the adhesive object are known, this information mayfacilitate (e.g., increase an accuracy and/or speed of) the imageregistration. By having a known dimension, the portion of the imagewhich includes the adhesive object, or parts thereof, can be subtractedout from the scanned object, thereby leaving behind the originalobject's shape without the presence of the reference adhesive object.Additionally, if the shape, size, pattern, color, etc. of the adhesiveobject are known, then even if only a portion of the adhesive object iscaptured in an intraoral image, a shape, pattern, size, color, etc. ofthe remainder of the adhesive object may be extrapolated based on theknown properties of that adhesive object.

The adhesive object may also be a 3-dimensional (3D) object that isitself scanned as part of the process, such that the portions of theobject which are scanned in multiple images may be used as a commonreference object when trying to stitch the individual images togetherand create a singular image to be used for dental prosthesis ororthodontic appliance fabrication. The 3D object geometry can besubtracted from the 3D generated model.

FIG. 1 illustrates a flow diagram for a method 100 of performingintraoral scanning, in accordance with embodiments of the presentinvention. Some aspects of the method 100 may be performed by anoperator of an intraoral scanner (e.g., a dental practitioner), whileother aspects of method 100 are performed by processing logic thatcomprises hardware (e.g., circuitry, dedicated logic, programmablelogic, microcode, etc.), software (such as instructions run on aprocessing device), or a combination thereof. In one embodiment,processing logic is computing device 305 of FIG. 3.

At block 105 of method 100, one or more intraoral locations (e.g., of adental site) are determined to be candidates for application of adhesiveobjects. The one or more intraoral locations may be manually determinedby a dental practitioner, or may be determined by processing logic basedon information provided by the dental practitioner (e.g., based on anindication of which of a patient's teeth are missing). In oneembodiment, the dental practitioner uses a graphical user interface orother interface to indicate the patient's missing teeth. Processinglogic may then identify a target area of the dental site that containsprimarily or only soft tissue (e.g., edentulous regions). Such anedentulous region may prevent or complicate a successful intraoral scanof the patient because the soft tissue may lack distinctive features(e.g., geometrical features) having a definition that is suitable forperforming image registration (i.e. the tissue contours may be toosmooth to allow individual snapshots to be accurately registered to eachother). For example, soft tissue may not permit a surface shapemeasurement that is usable for accurate image registration. Theidentified target area may be part of a dental site that forms the focusof a particular dental procedure for the patient. It may be desirable toperform an intraoral scan of the target area to obtain 3D topographicalor surface data thereof (e.g., to form a 3D virtual model of the targetarea). The target area is typically part of a dental site on which aparticular procedure is to be performed, and in some cases may includefull mandibular or maxillary arches, or both arches such as would be thecase of full denture treatment and fully-edentulous implant treatments(including implant-supported dentures).

The manner in which the oral cavity is to be scanned may depend on theprocedure to be applied thereto. For example, if an upper or lowerdenture is to be created, then a full scan of the mandibular ormaxillary edentulous arches may be performed. In contrast, if a bridgeis to be created, then just a portion of a total arch may be scannedwhich includes the edentulous region, the neighboring abutment teeth andthe opposing arch and dentition. Thus, the dental practitioner may alsoinput the identity of a procedure to be performed. For this purpose, thedental practitioner may choose the procedure from a number of presetoptions on a drop-down menu or the like, from icons or via any othersuitable graphical input interface. Alternatively, the identity of theprocedure may be input in any other suitable way, for example by meansof preset code, notation or any other suitable manner, processing logichaving been suitably programmed to recognize the choice made by theuser. By way of non-limiting example, dental procedures may be broadlydivided into prosthodontic (restorative) and orthodontic procedures, andthen further subdivided into specific forms of these procedures. Theterm prosthodontic procedure refers, inter alia, to any procedureinvolving the oral cavity and directed to the design, manufacture orinstallation of a dental prosthesis at a dental site within the oralcavity, or a real or virtual model thereof, or directed to the designand preparation of the dental site to receive such a prosthesis. Aprosthesis may include any restoration such as crowns, veneers, inlays,onlays, and bridges, for example, and any other artificial partial orcomplete denture. The term orthodontic procedure refers, inter alia, toany procedure involving the oral cavity and directed to the design,manufacture or installation of orthodontic elements at a dental sitewithin the oral cavity, or a real or virtual model thereof, or directedto the design and preparation of the dental site to receive suchorthodontic elements. These elements may be appliances including but notlimited to brackets and wires, retainers, clear aligners, or functionalappliances.

A type of scanner to be used may also be input, typically by a dentalpractitioner choosing one among a plurality of options. If the scannerthat is being used is not recognizable by processing logic, it maynevertheless be possible to input operating parameters of the scannerthereto instead. For example, the optimal spacing between a head of thescanner and scanned surface can be provided, as well as the capture area(and shape thereof) of the dental surface capable of being scanned atthis distance. Alternatively, other suitable scanning parameters may beprovided.

Processing logic may identify spatial relationships that are suitablefor scanning the dental site so that complete and accurate image datamay be obtained for the procedure in question. Processing logic mayestablish an optimal manner for scanning a target area of the dentalsite. Further, processing logic may compute an optimal placement for theadhesive object (or objects) based on the missing teeth or on otherinformation. Processing logic may then identify to a dental practitionerone or more locations (e.g., the optimal placement) at which an adhesiveobject is recommended based on the indication by the dental practitionerof the patient's missing teeth or based on other information. Processinglogic then may identify to the dental practitioner particular locationsto place adhesive objects and/or a placement pattern for multipleadhesive objects. Processing logic may take into consideration a fieldof view of an intraoral scanner to be used when recommending placementlocations for the adhesive objects to ensure that image registrationwill be successful.

Having identified the target area and/or locations for placement ofadhesive objects, a scanning protocol is identified or determined byrelating the type of scanner, resolution thereof, capture area at anoptimal spacing between the scanner head and the dental surface to thetarget area, etc. For a point-and-shoot scanning mode, the scanningprotocol comprises a series of scanning stations spatially associatedwith the dental surfaces of the target area. Preferably, overlapping ofthe images or scans capable of being obtained at adjacent scanningstations is designed into the scanning protocol to enable accurate imageregistration, so that intraoral images can be stitched together toprovide a composite 3D virtual model. For a continuous scanning mode(video scan), scanning stations may not be identified. Instead, apractitioner may activate the scanner and proceed to move the scannerwithin the oral cavity to capture a video of the target area frommultiple different viewpoints.

At block 110, the dental practitioner affixes one or more adhesiveobjects to the patient at the identified locations of the dental site(e.g., to the portions of a target area that are missing teeth). Thismay include stamping an ink onto the identified regions, formingextrusions onto the identified regions, placing stickers onto theidentifier regions, or placing other adhesive objects onto theidentified regions. Each adhesive object will act as a geometricreference point for image registration. The adhesive objects arediscussed in greater detail below.

Some embodiments are described herein with reference to placing adhesiveobjects to dental sites that are missing one or more teeth. However, itshould be noted that adhesive objects described herein may also be usedin areas with teeth to improve the accuracy of the scan, increasescanning speed and/or eliminate incorrect registration. In these cases,the intraoral objects can be adhered to the soft tissue on either orboth sides of the teeth or on the teeth themselves. Accordingly, itshould be understood that embodiments described herein with regards tothe placement and use of adhesive objects may equally apply toedentulous regions, other dental sites that are missing one or moreteeth, and dental sites that include teeth.

At block 115, the dental practitioner performs an intraoral scan of thedental site including the target area and the one or more adhesiveobjects. This may include performing an intraoral scan of a partial orfull mandibular or maxillary arch, or a partial or full scan of botharches. Performing the intraoral scan in particular may includegenerating a first intraoral image having a first representation of atleast one portion of an adhesive object (block 120) and generating asecond intraoral image having a second representation of at least aportion of the adhesive object (block 125). The first and secondintraoral images may be discrete images (e.g., taken from apoint-and-shoot mode) or frames of an intraoral video (e.g., taken in acontinuous scanning or video mode). In one embodiment, the secondrepresentation is of the same portion of the object that is shown in thefirst representation. Alternatively, the second representation may be ofa different portion of the object than is shown in the firstrepresentation. In the second instance, these images may be stitched ifprocessing logic has access to information identifying the shape,pattern and/or size of the adhesive object. Each intraoral image may bea three dimensional (3D) image having a particular height, width anddepth. In some embodiments, an intraoral scanner is used that generates3D images having a depth of 12-14 mm, a height of 13-15 mm and a widthof 17-19 mm (e.g., a depth of 13 mm, height of 14 mm and width of 18 mmin one particular embodiment).

At block 128, processing logic receives the intraoral images generatedfrom the intraoral scan. At block 130, processing logic then registers(i.e., “stitches” together) the intraoral images generated from theintraoral scan. This may include registering the first intraoral imageto the second intraoral image using at least a portion of the firstrepresentation of the adhesive object in the first intraoral image andat least one portion (which may be an equivalent portion of a differentportion) of the second representation of the adhesive object in thesecond intraoral image (block 135). In one embodiment, performing imageregistration includes capturing 3D data of various points of a surfacein multiple images (views from a camera), and registering the images bycomputing transformations between the images. The images may then beintegrated into a common reference frame by applying appropriatetransformations to points of each registered image. Image registrationis discussed in greater detail with reference to FIG. 2.

At block 140, processing logic then generates a 3D model of the targetarea based on the registration. The 3D model may be a virtual or digitalmodel showing the surface features of the target area. For a 3D model ofa full dental arch, the arch width of the 3D model may be accurate towithin 200 microns of the arch width of the patient's actual dentalarch.

In one embodiment, after the 3D model of the dental site is generated,representations of the adhesive object (or objects) are subtracted fromthe 3D model (e.g., via Boolean subtraction). Since the properties(e.g., shape, size, color, surface pattern, etc.) of the adhesive objectare known, the representation may be subtracted using these properties.This may leave a gap or empty region on a surface of the 3D model, whichmay be filled in by interpolating between portions of the dental sitesurrounding the gap as well as by filling with a surface parallel to thebase of the adhesive object. This allows the underlying dental surfaceto be estimated with reasonable accuracy. Alternatively, a subsequentscan of the underlying dental surface may be added to fill in themissing region by using another registration object placed elsewhere inthe arch at a different location, in order to accurately orient andposition the intraoral surfaces previously covered up by theregistration marker.

FIG. 2 illustrates a flow diagram for a method 200 of performing imageregistration of intraoral images, in accordance with embodiments of thepresent invention. The method 200 is performed by processing logic thatcomprises hardware (e.g., circuitry, dedicated logic, programmablelogic, microcode, etc.), software (such as instructions run on aprocessing device), or a combination thereof. In one embodiment, method200 is performed by a computing device, such as computing device 305 ofFIG. 3. In one embodiment, method 200 is performed at block 130 ofmethod 100.

At block 205 of method 200, processing logic receives image data from anintraoral scan of a patient. The image data includes multiple digitalintraoral images of a target zone of the dental site. In one embodiment,the multiple digital intraoral images are discrete images. In oneembodiment, the multiple digital images are part of a video (e.g., acontinuous scan) taken of the dental site. The video may be taken whilea dental practitioner moves the intraoral scanner between points withinthe patient's oral cavity. The image data may be received from anintraoral scanner, from a remote computing device, from a local orremote data store, or from another source.

At block 210, processing logic receives an indication that adhesiveobjects were used as geometric reference points. For example, the dentalpractitioner may select from a drop down menu that adhesive objects havebeen affixed to a target area of a dental site. The dental practitionermay further select specific types of adhesive objects that have beenused (e.g., identify particular adhesive objects having specific shapes,sizes, colors, patterns, thicknesses, and so forth). Options fordifferent types of adhesive objects may be presented to the dentalpractitioner in a graphical user interface (e.g., in drop down menus) orvia a command line interface. Alternatively, the dental practitioner maymanually input the type of adhesive object (e.g., by typing in one ormore parameters of the adhesive objects). In one embodiment, each typeof adhesive object is associated with a particular identifier (ID), andthe dental practitioner may indicate a specific type of adhesive objectby selecting or inputting a particular ID.

In one embodiment, at block 215, processing logic determines whether anidentification of a specific type of adhesive object (or multiple typesof adhesive objects) have been used. If a specific adhesive object orobjects are identified, the method may continue to block 220. Otherwise,the method may proceed to block 225. If the operations of block 215 arenot performed, then the method may proceed from block 210 to block 225.

At block 220, processing logic compares one or more of the receivedintraoral images to known properties (parameters) of the specificadhesive object or objects that were specified. The known properties maybe stored in a data store that is accessible to processing logic. Basedon such a comparison, processing logic may identify representations ofthe adhesive object in one or more of the intraoral images.

At block 225, processing logic compares received intraoral images toknown properties of various adhesive objects to identify specificadhesive objects in one or more intraoral images. The computationperformed at block 225 may be greater than the computation performed atblock 220 since a greater number of comparisons are performed.Additionally, processing logic may be better able to identify anadhesive object from a distorted image when it is searching for aparticular adhesive object that is known to have been used.

Some adhesive objects may have specific colors and/or surface patternsthat may be used to detect the adhesive object and/or to determine anorientation of the adhesive object. However, conventional intraoralscanners are configured to detect 3D surfaces, but not to detect colorsor two-dimensional (2D) patterns. In contrast, in some embodimentsprocessing logic includes color recognition algorithms, patternrecognition algorithms, and/or object detection algorithms that rely oncolor and/or texture to enable detection of such adhesive objects. Inone embodiment if a dental practitioner has indicated adhesive objectsthat rely on a 3D shape to be detected, color and pattern recognitionalgorithms, for example, may be disabled by processing logic. However,if the dental practitioner has indicated an adhesive object that relieson color and/or surface pattern for detection, or if the dentalpractitioner has failed to indicate any specific adhesive objects, suchcolor recognition, pattern recognition and/or object detectionalgorithms may be enabled and used to identify adhesive objects.Applying such a color recognition, pattern recognition or objectdetection algorithm may include analyzing the intraoral images toidentify a known color and/or known pattern of an adhesive object. Oncethe known color or pattern are identified, then a representation of aparticular adhesive object having the known color and/or pattern may beidentified in an image, and may be recognized as an instance of theadhesive object.

In one embodiment, at block 230 processing logic determines whetherportions of adhesive objects have been identified (e.g., whether lessthan an entirety of an adhesive object has been identified in anintraoral image). If a partial adhesive object is identified, then themethod may continue to block 235, at which processing logic may generatea representation of the full adhesive object in the intraoral image. Forexample, if the image includes one corner of an adhesive object,processing logic may replace the corner with a representation of thefull adhesive object. This may improve image registration betweenimages. For example, if two different images contain representations ofdifferent non-overlapping portions of an adhesive object, then it mayimpair an ability of processing logic to determine a propertransformation of the images to align them. However, if the shape of theadhesive object is known, and can be added to the intraoral images, thenthe images will include overlapping portions of the adhesive object thatcan be used to compute transformation vectors, and thus facilitateregistration of the two intraoral images.

At block 240, processing logic performs image registration between theintraoral images. In particular, processing logic uses representationsof the adhesive objects for at least two of the intraoral images tocompute transformations that will cause the adhesive objects from thetwo intraoral images to be aligned. In one embodiment, processing logicperforms image registration in a manner discussed in U.S. Pat. No.6,542,249, issued Apr. 1, 2003, which is incorporated herein byreference.

In one embodiment, image registration is performed for each pair ofadjacent or overlapping intraoral images (e.g., each successive frame ofan intraoral video). Image registration algorithms are carried out toregister two adjacent intraoral images, which essentially involvesdetermination of the transformations which align one image with theother. Each registration between a pair of images may be accurate towithin 10-15 microns. Image registration may involve identifyingmultiple points in each image (e.g., point clouds) of an image pair,surface fitting to the points of each image, and using local searchesaround points to match points of the two adjacent images. For example,processing logic may match points of one image with the closest pointsinterpolated on the surface of the other image, and iteratively minimizethe distance between matched points. Processing logic may also find thebest match of curvature features at points of one image with curvaturefeatures at points interpolated on the surface of the other image,without iteration. Processing logic may also find the best match ofspin-image point features at points of one image with spin-image pointfeatures at points interpolated on the surface of the other image,without iteration. Other techniques that may be used for imageregistration include those based on determining point-to-pointcorrespondences using other features and minimization ofpoint-to-surface distances, for example. Other image registrationtechniques may also be used.

Many image registration algorithms perform the fitting of a surface tothe points in adjacent images, which can be done in numerous ways.Parametric surfaces such as Bezier and B-Spline surfaces are mostcommon, although others may be used. A single surface patch may be fitto all points of an image, or alternatively, separate surface patchesmay be fit to any number of a subset of points of the image. Separatesurface patches may be fit to have common boundaries or they may be fitto overlap. Surfaces or surface patches may be fit to interpolatemultiple points by using a control-point net having the same number ofpoints as a grid of points being fit, or the surface may approximate thepoints by using a control-point net which has fewer number of controlpoints than the grid of points being fit. Various matching techniquesmay also be employed by the image registration algorithms.

In one embodiment, processing logic may determine a point match betweenimages, which may take the form of a two dimensional (2D) curvaturearray. A local search for a matching point feature in a correspondingsurface patch of an adjacent image is carried out by computing featuresat points sampled in a region surrounding the parametrically similarpoint. Once corresponding point sets are determined between surfacepatches of the two images, determination of the transformation betweenthe two sets of corresponding points in two coordinate frames can besolved. Essentially, an image registration algorithm may compute atransformation between two adjacent images that will minimize thedistances between points on one surface, and the closest points to themfound in the interpolated region on the other image surface used as areference.

Processing logic repeats image registration for all adjacent image pairsof a sequence of intraoral images to obtain a transformation betweeneach pair of images, to register each image with the previous one.Processing logic then integrates all images into a single 3D model byapplying the appropriate determined transformations to each of theimages. Each transformation may include rotations about one to threeaxes and translations within one to three planes.

FIG. 3 illustrates one embodiment of a system 300 for performingintraoral scanning and/or generating a virtual three dimensional modelof a dental site. In one embodiment, system 300 carries out one or moreoperations of above described method 200 and/or method 300. System 300includes a computing device 305 that may be coupled to a scanner 350and/or a data store 310.

Computing device 305 may include a processing device, memory, secondarystorage, one or more input devices (e.g., such as a keyboard, mouse,tablet, and so on), one or more output devices (e.g., a display, aprinter, etc.), and/or other hardware components. Computing device 305may be connected to a data store 310 either directly or via a network.The network may be a local area network (LAN), a public wide areanetwork (WAN) (e.g., the Internet), a private WAN (e.g., an intranet),or a combination thereof. The computing device and the memory device maybe integrated into the scanner in some embodiments to improveperformance and mobility.

Data store 310 may be an internal data store, or an external data storethat is connected to computing device 305 directly or via a network.Examples of network data stores include a storage area network (SAN), anetwork attached storage (NAS), and a storage service provided by acloud computing service provider. Data store 310 may include a filesystem, a database, or other data storage arrangement.

In some embodiments, a scanner 350 for obtaining three-dimensional (3D)data of a dental site in a patient's oral cavity is also operativelyconnected to the computing device 305. Scanner 350 may include a probe(e.g., a hand held probe) for optically capturing three dimensionalstructures (e.g., by confocal focusing of an array of light beams). Oneexample of such a scanner 350 is the iTero® intraoral digital scannermanufactured by Align Technology, Inc. Other examples of intraoralscanners include the 3M™ True Definition Scanner and the Apollo DIintraoral scanner and CEREC AC intraoral scanner manufactured bySirona®.

The scanner 350 may be used to perform an intraoral scan of a patient'soral cavity. A result of the intraoral scan may be a sequence ofintraoral images that have been discretely generated (e.g., by pressingon a “generate image” button of the scanner for each image).Alternatively, a result of the intraoral scan may be one or more videosof the patient's oral cavity. An operator may start recording the videowith the scanner 350 at a first position in the oral cavity, move thescanner 350 within the oral cavity to a second position while the videois being taken, and then stop recording the video. In some embodiments,recording may start automatically as the scanner identifies either teethor adhesive objects In either case, the scanner 350 may transmit thediscrete intraoral images or intraoral video (referred to collectivelyas image data 335) to the computing device 305. Note that in someembodiments the computing device may be integrated into the scanner 350.Computing device 305 may store the image data 335 in data store 310.Alternatively, scanner 350 may be connected to another system thatstores the image data in data store 310. In such an embodiment, scanner350 may not be connected to computing device 305.

Computing device 305 may include an adhesive object identificationmodule 315, an image registration module 320, and a model generationmodule 325. Adhesive object identification module 315 may analyzereceived image data 335 using known adhesive object properties 330 toidentify adhesive objects in the intraoral images of the image data 335.The adhesive object properties 330 may be stored in data store 310. Inone embodiment, user input identifies specific adhesive objects thathave been used, reducing a complexity of the search for adhesiveobjects. Image registration module 320 registers the intraoral imagesusing the previously described image registration techniques. In oneembodiment, image registration module 320 uses an output of the adhesiveobject identification module 315 when performing the image registration.In other embodiments (e.g., where only 3D adhesive objects with nosurface pattern are used), image registration module 320 may performimage registration without using any stored or known information aboutthe adhesive objects. Additionally, in some embodiments adhesive objectidentification module 315 is not included in computing device 305. Afterimage registration is complete, or as image registration is performed,model generation module 325 generates a 3D virtual model of the imageddental site.

FIG. 4A illustrates an example dental arch 400 that is missing twoteeth. The illustrated dental arch 400 is missing a pair of adjacentteeth at a target zone 415 of the dental arch 400. The dental arch 400includes gums 405 and multiple teeth 410.

Two intraoral images 420, 425 have been taken of the target zone 415 ofthe dental arch 400. Each of the intraoral images 420, 425 may have beengenerated by an intraoral scanner having a particular distance from thedental surface being imaged (e.g., from the dental arch 400). At theparticular distance, the intraoral images 420, 425 have a particularscan area and scan depth. The shape and size of the scan area willgenerally depend on the scanner, and is herein represented by arectangle. Each image may have its own reference coordinate system andorigin. Each intraoral image may be generated by a scanner at aparticular position (scanning station). The location and orientation ofscanning stations may be selected such that together the intraoralimages adequately cover the entire target zone 415. Preferably, scanningstations are selected such that there is overlap between the intraoralimages 420, 425 as shown. Typically, the selected scanning stations willdiffer when different scanners are used for the same target area,depending on the capture characteristics of the scanner used. Thus, ascanner capable of scanning a larger dental area with each scan (e.g.,having a larger field of view) will use fewer scanning stations than ascanner that is only capable of capturing 3D data of a relativelysmaller dental surface. Similarly, the number and disposition ofscanning stations for a scanner having a rectangular scanning grid (andthus providing projected scanning areas in the form of correspondingrectangles) will typically be different from those for a scanner havinga circular or triangular scanning grid (which would provide projectedscanning areas in the form of corresponding circles or triangles,respectively).

As shown, there are no teeth in the target zone. This lack of teethmakes registration between the two intraoral images 420, 425 challengingbecause the smooth surfaces when optically captured as partial scans aredifficult to accurately stitch together without distinctive registrationfeatures within the field of each partial scan.

FIG. 4B illustrates the oral cavity 400 of FIG. 4A with the addition ofan adhesive object 430 placed at the location of the missing teeth, inaccordance with embodiments of the present invention. The adhesiveobject 430 may be placed anywhere on the gum at or near a target area.In one embodiment, the adhesive object is placed at (or approximatelyat) a gum line near the target area (or where the gum line would be ifthe target area had teeth). The same intraoral images 420, 425 of thetarget zone 415 from FIG. 4A are shown, with the exception that theintraoral images include representations of at least a portion of theadhesive object 430.

The adhesive object 430 may be made of a body having a particular shape,an upper surface and a lower surface. The adhesive object may becomposed of an edible substance such as a carbohydrate-based substance.Alternatively, the adhesive object may be a pattern of ink that has beenstamped or temporarily tattooed onto gingival surfaces at the locationof the missing teeth. In one embodiment, the body is composed of amaterial that does not substantially change its shape (or does notchange its shape at all) during intraoral scanning. For example, thebody may not degrade or wash away (e.g., by saliva) within the timeframe of performing the intraoral scan. In the example of an ink, theink may retain its pattern during the time frame of the intraoral scan.

In one embodiment, the adhesive object includes a body and an adhesiveon a lower surface of the body. In one embodiment, the body is composedof a different substance than the adhesive on the lower surface of thebody. In one embodiment, the adhesive is a dry liquid-activated adhesive(e.g., a dry water-activated adhesive). One example of a drywater-activated adhesive that may be used is sodium alginate. Otheradhesives that may be used include cellulose gum, sodium carboxymethylcellulose, methyl cellulose, polyvinyl methyl ether maleate, gelatin,pectin, karaya, tragacanth, and so on. Any of these adhesives may beused on its own, or as a mixture with other adhesives (e.g., as amixture with sodium alginate). In one embodiment, the adhesive materialincludes a thermoplastic ethylene oxide polymer film having a drywater-activated adhesive dissolved and dispersed in the polymer film.For example, any of the aforementioned adhesives may be mixed with anethylene oxide polymer to form a bonding agent. The water-activatedadhesive may include from 0% to approximately 90% by weight of anethylene oxide polymer/water-activated adhesive mixture, for example. Inone embodiment, the adhesive is a film of ethylene oxide sandwichedbetween superimposed fiber-faced webs. In one particular embodiment, theadhesive includes a combination of mannitol, hydroxypropul cellulose,polyethylene glycol and magnesium stearate. The adhesive may include 60grams mannitol as a filler, 20 grams hydroxypropyl cellulose as abinder, 18.9 grams polyethulene glycol 800-1000 as a binder, and 1 grammagnesium stearate as a lubricant. In another embodiment, the adhesiveincludes a combination of mannitol, carbopol, polyethylene glycol andmagnesium stearate. The adhesive may include 50 grams mannitol as afiller, 20 grams carbopol 934 as a binder, 28.9 grams polyethuleneglycol 800-1000 as a binder, and 1 gram magnesium stearate as alubricant.

The adhesive object may include a feature on the upper surface of thebody that is detectable by an intraoral scanner. Alternatively, in thecase of a stamped adhesive object the adhesive object may include afeature stamped directly to a gingival surface. At least one of theshape of the body or the feature on the upper surface provides ageometrical or optical reference point for image registration of imagesgenerated by the intraoral scanner.

FIG. 7 illustrates various examples of adhesive objects. In oneembodiment, the feature on the upper surface of the adhesive objectsincludes a pattern and/or a color (or multiple colors). Examples ofpatterns that may be used include a checkerboard pattern (e.g.,alternating black and white squares), a line pattern, a grid pattern, a2-D barcode pattern, a random dot pattern, and so on. For example,adhesive object 705 has a star shape and includes a first grid patternon an upper surface. Adhesive object 710 has a star shape and includes aline pattern on an upper surface. Adhesive object 715 has a star shapeand includes a second grid pattern on an upper surface. Adhesive object720 has a star shape and includes a checkerboard pattern on an uppersurface.

Any shape may be used for the adhesive object. For example, adhesiveobjects 705-720 are five-pointed stars. Adhesive object 725 is apentagon. Adhesive object 730 is a six-pointed star. Adhesive object 735is a seven-pointed star. Adhesive object 740 is a circle.

In one embodiment, the adhesive object is substantially flat (e.g.,approximately two-dimensional). For example, the adhesive object mayhave a flat upper surface and a flat lower surface, and may have athickness of less than 100 microns in one embodiment. Adhesive objects705-740 may be examples of such flat adhesive objects. In oneembodiment, the adhesive object is a pattern of ink stamped onto agingival surface. The ink may not be susceptible to erosion from thegingival surface within a time frame used to perform an intraoral scan.

In one embodiment, the adhesive object is substantially flat, but has athickness that is detectable to the intraoral scanner. Thus, theadhesive object may be detected as a three dimensional object. In oneembodiment, the adhesive object has a thickness of greater than 20microns. Adhesive object 760 has a cylindrical shape, and may have athickness of 20 microns or greater. Adhesive object 750 has arectangular shape, and may have a thickness of 20 microns or greater. Inone embodiment, adhesive object 760 and/or adhesive object 750 have athickness of 0.5 mm or greater. Note that any of the example adhesiveobjects 705-740 may also have such thicknesses.

In one embodiment, adhesive object has three dimensional surfacefeatures, such as multiple edges, sides, facets, etc. that aredetectable to an intraoral scanner. For example, adhesive object 745 hasthe shape of a cone, and adhesive object 755 has the shape of a pyramid.

Though colors are not shown, any of the example adhesive objects mayhave solid colors or patterns with various different colors. It shouldbe noted that the illustrated examples of adhesive objects isnon-exhaustive, and that any combination of shapes, sizes and/orpatterns may be used for adhesive objects. Examples of heights andwidths that may be used for adhesive objects are anywhere from 1 mm to10 mm, though other heights and widths may also be used. It should alsobe noted that the adhesive objects may be rigid or non-rigid. The shapesof non-rigid adhesive object may change based on the surface that theyare affixed to. For example, a stamp on a non-flat gingival surface or anon-rigid sticker on a non-flat gingival surface may be slightlydistorted due to a curvature of the gingival surface.

The adhesive objects may be individual adhesive objects as shown in FIG.7. Such adhesive objects may be distributed in the form of sheets (e.g.,sheets of stickers). In one example, the adhesive objects are on a thinlong sheet that is approximately wide enough to hold a single adhesiveobject and long enough to hold many adhesive objects. The sheet may beprovided in a roll, which may be inserted into a dispenser. The adhesiveobjects in the roll may have the same shape, size and/or pattern, ordifferent shapes, sizes and/or patterns. If different shapes, sizes,patterns, etc. are used for the different adhesive objects in the roll,then the sequence may be predetermined and repeatable. For example, thesequence may repeat as a five pointed star, a circle, and a six pointedstar, followed by another five pointed star, circle, six pointed star,and so on. In such an embodiment, a computing device that performs imagerecognition may, after identifying a particular adhesive object, predictadjacent adhesive objects. In one embodiment, the dispenser may have awheel at one end, which may be rolled along a dental site to depositadhesive objects at a predetermined spacing.

In one embodiment, the adhesive objects are connected together by links,as shown in FIG. 8. Accordingly, an adhesive object may include multiplecomponents that each provide a distinct geometrical reference point, themultiple components being interconnected by a plurality of links. Eachcomponent in such a chain may be considered as a separate adhesiveobject. The links may have a length that ensures that the separationbetween adjacent adhesive objects will not be greater than the field ofview of an intraoral scanner that is to be used. For example, if anintraoral scanner to be used has a field of view with a width of 18 mm,then the links may be shorter than 18 mm. Therefore, intraoral imageswith adjacent or overlapping fields of view may contain representationsof the same adhesive object to enable accurate image registration.

Alternatively, a single long narrow adhesive object may be provided(e.g., such as a tape in a roll) that has many different shapesimprinted on it. The long narrow adhesive object may have a width of1-10 mm and a length of anywhere from 10-100 mm in some embodiments.Varying patterns and/or colors may also be printed on the long narrowadhesive object in addition to or instead of 3D shapes. The shapes (orcolors and/or patterns) may be separated by a regular interval (e.g.,every 5 mm, every 10 mm, etc.). Alternatively, the shapes may beseparated by varying distances. In one embodiment, only one or a fewtypes of shapes are included in the adhesive object, but there is avarying (e.g., pseudorandom) spacing between the shapes. Alternatively,many (e.g., up to tens to hundreds of) distinct shapes are included inthe adhesive object. In one embodiment, each of the shapes is selectedso as to have minimal (if any) similar features to the other shapes. Forexample, a crescent and a cross may be used together because they haveno similar features. In one embodiment, the adhesive object is providedas a roll, and a practitioner cuts the roll to create an adhesive objectwith a custom length that may be sufficient to span an edentulousregion. Such an adhesive object may be applied to an edentulous regionto span approximately the whole edentulous region.

FIG. 8 shows a first chain 800 of adhesive objects, which includesadhesive objects 805-820, each of which has the same shape, size andpattern. These adhesive objects 805-820 are interconnected via links825. Links 825 may be made up of an edible substance (e.g., a sugarbased substance). FIG. 8 additionally shows a second chain 850 ofadhesive objects, which includes adhesive objects 855-870. The secondchain 850 includes an example repeating sequence of different adhesiveobjects. The sequence includes a five pointed star, a circle, and a sixpointed star, which repeats. As shown, adhesive object 855 is a fivepointed star, adhesive object 860 is a circle, and adhesive object 865is a six pointed star, and adhesive object 870 is another five pointedstar, staring the sequence over. The first chain 800 and second chain850 may be distributed in a roll, which may be usable in a dispenser, inone embodiment.

In another embodiment, the adhesive objects are distributed as a stackof adhesive objects (e.g., in a capsule). The adhesive objects in thestack may have the same shape or a different shape. The stack ofadhesive objects may be inserted into a dispenser or applicator, whichmay resemble a pen or syringe in some embodiments. A dental practitionermay place a tip of the applicator onto a dental site and depress abutton or plunger, for example, which may cause a single adhesive objectto be placed at the dental site. When the applicator empties, it may berefilled with a new stack of adhesive objects. Alternatively, stacks ofadhesive objects may be distributed in disposable applicators.

In one embodiment, adhesive objects are initially in a liquid or gelstate, and harden into a rigid or semi-rigid adhesive object uponplacement at a dental site or upon light or chemical cure. For example,adhesive objects may be composed of a material that swells, hardens, andadheres to a surface when it comes into contact with water. One possibleadhesive that adheres responsive to a light or chemical cure is abis-glycidyl methacrylate (bis-GMA) based resin. A pen or syringe typeapplicator may have a tip with an opening having a particular shape(e.g., a star shaped opening). A dental practitioner may place a tip ofthe applicator against a dental site and depress a button or plunger,which may cause an amount of liquid or gel to be extruded through theopening onto the dental site. The gel or liquid may form an adhesiveobject having the shape of the opening at the applicator's tip, and mayharden and adhere on contact with the dental site or upon activation bylight or chemical cure. The size of the extruded adhesive object may bedependent on a size of the opening and on an amount of gel or liquidthat is extruded.

Referring back to FIG. 4B, an adhesive object 430 has been placed in aparticular target zone 415 of a dental arch 400 to facilitate imageregistration. The adhesive object 430 may be any of the aforementionedadhesive objects. A first intraoral image 420 and a second intraoralimage 425 are taken of the target area 415, and each include arepresentation of the adhesive object 430.

FIG. 5A, illustrates first intraoral image 420, which includes a firstrepresentation of adhesive object 430, in accordance with one embodimentof the present invention. FIG. 5B illustrates second intraoral image425, which includes a second representation of adhesive object 430, inaccordance with one embodiment of the present invention. As shown, thefirst representation of adhesive object 430 in the first intraoral image420 is of only a portion of the adhesive object 430. This partialrepresentation is sufficient for image registration.

FIG. 5C illustrates a successful image registration of the firstintraoral image 420 of FIG. 5A and the second intraoral image 425 ofFIG. 5B, in accordance with one embodiment of the present invention. Asshown, transformations have been performed in one or both of the firstintraoral image 420 and second intraoral image 425 to cause the firstrepresentation of the adhesive object 430 to line up with the secondrepresentation of the adhesive object 430. Such transformations mayinclude rotations around up to three axes and translations in up tothree planes.

FIG. 5D illustrates a first intraoral image 520, which includes a firstrepresentation of a first portion of an adhesive object 530, inaccordance with one embodiment of the present invention. FIG. 5Eillustrates a second intraoral image 525, which includes a secondrepresentation of a second portion of the adhesive object 530, inaccordance with one embodiment of the present invention. As shown, thefirst representation of adhesive object 530 and the secondrepresentation of the adhesive object 530 each include a differentportion of the adhesive object. This may cause image registrationbetween these two images to be difficult. However, in some embodimentsthe system that performs the image registration includes informationidentifying the shape, size, pattern, and/or other information of theadhesive object. This enables the system to extrapolate the shape andorientation of the rest of the adhesive object to form a virtualrepresentation of the adhesive object 532 from each of therepresentations of the adhesive object 530. The virtual representation532 can be used to successfully stitch the two intraoral imagestogether, as shown in FIG. 5F, even though there may be no or minimaloverlap between the intraoral images 520, 525.

FIG. 5G illustrates a typodont 505, in accordance with one embodiment.The typodont 505 includes adhesive objects 510-514 disposed around thetypodont 505. Alternatively, the adhesive objects may be affixeddirectly to the typodont. The adhesive object may be a one or more longthin bodies with multiple patterns or shapes imprinted thereon as shown.Alternatively, multiple distinct adhesive objects of varying shapesand/or sizes may be used. For example, as shown different shapes mayinclude a T shape, a triangle shape, a C shape, a vertical bar shape, adiagonal bar shape, a diamond shape, a square shape, and so on. In oneembodiment, adjacent adhesive objects have different shapes.

FIG. 5H illustrates a reconstruction 555 of the typodont 505. Anintraoral scan including multiple discrete images or a video of thetypodont 505 has been taken. The images were stitched together and usedto construct a virtual model including the reconstructed typodont 555and reconstructed adhesive objects 560-564. In one embodiment, as shown,the adhesive objects have multiple different shapes. Use of adhesiveobject with varying shapes may improve an accuracy of imageregistration. An intraoral scan performed without the adhesive objectswould be unsuccessful because there are insufficient surface features tostitch images together. Similar results are achieved for scans ofedentulous regions of an oral cavity.

FIG. 6A illustrates an edentulous arch 600 with the addition of apattern of adhesive objects 630-640 placed at various locations, inaccordance with embodiments of the present invention. The placement ofthe adhesive objects at various locations along the edentulous arch 600enables the edentulous arch 600 to be accurately modeled using intraoralscanning techniques. The placement pattern of the adhesive objects maybe determined by a dental practitioner, or may be determined by acomputing device based on a field of view of a scanner to be used. Asequence of intraoral images 615 may be taken, which when taken togethermay depict the entire edentulous arch 600. These intraoral images may beregistered with one another using representations of the adhesiveobjects 630-640 to generate a 3D virtual model or simulation of theedentulous arch 600. This 3D virtual model may be used to manufacturedentures and/or implant abutments. In one example, the generated 3Dvirtual model of the edentulous arch 600 may be used to develop atreatment plan for an all-on-4® dental implant-supported denture. Theall-on-4 dental implant uses four implants abutments (with underlyingimplants) to support a fixed prosthesis representing 12 to 14 teeth.

FIG. 6B illustrates an edentulous arch 650 with the addition of apattern of adhesive objects 670-680 placed at various locations, inaccordance with embodiments of the present invention. The placement ofthe adhesive objects at various locations along the edentulous arch 650enables the edentulous arch 650 to be accurately modeled using intraoralscanning techniques. A sequence of intraoral images 665 may be taken,which when taken together may depict the entire edentulous arch 650.These intraoral images may be registered with one another usingrepresentations of the adhesive objects 670-680 to generate a 3D virtualmodel or simulation of the edentulous arch 650. As shown in FIG. 6B,multiple different shapes, sizes and/or patterns of adhesive objects maybe used together. For example, a first adhesive object 670 may be a fivepointed star, a second adhesive object 671 may be a six pointed star, athird adhesive object 672 may be a pentagon, a fourth adhesive object672 may be a triangle, a fifth adhesive object 674 may be a cross, andso on. Use of multiple different shapes may improve an accuracy of imageregistration in some embodiments.

Adhesive objects may be applied by an intraoral applicator. In oneembodiment, the intraoral applicator applies temporary tattoo or inkbased adhesive objects. The intraoral applicator may be a pen typeapplicator having a tip with a particular shape that has been carved,molded, laser engraved or vulcanized into the tip. The tip may be arubber tip that is pressed into an ink well to collect ink and thenpressed against a gingival surface to transfer the ink to the gingivalsurface. The transferred ink will have the particular shape of the tip.In another embodiment, the intraoral applicator applies other types ofadhesive objects such as stickers onto a gingival surface.

FIG. 9 illustrates a diagrammatic representation of a machine in theexample form of a computing device 900 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeembodiments, the machine may be connected (e.g., networked) to othermachines in a Local Area Network (LAN), an intranet, an extranet, or theInternet. The machine may operate in the capacity of a server or aclient machine in a client-server network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine may be a personal computer (PC), a tablet computer, a set-topbox (STB), a Personal Digital Assistant (PDA), a cellular telephone, aweb appliance, a server, a network router, switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines (e.g., computers)that individually or jointly execute a set (or multiple sets) ofinstructions to perform any one or more of the methodologies discussedherein.

The example computing device 900 includes a processing device 902, amain memory 904 (e.g., read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) such as synchronous DRAM (SDRAM), etc.), astatic memory 906 (e.g., flash memory, static random access memory(SRAM), etc.), and a secondary memory (e.g., a data storage device 928),which communicate with each other via a bus 908.

Processing device 902 represents one or more general-purpose processorssuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processing device 902 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,processor implementing other instruction sets, or processorsimplementing a combination of instruction sets. Processing device 902may also be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. Processing device 902 is configured to execute theprocessing logic (instructions 926) for performing operations and stepsdiscussed herein.

The computing device 900 may further include a network interface device922 for communicating with a network 964. The computing device 900 alsomay include a video display unit 910 (e.g., a liquid crystal display(LCD) or a cathode ray tube (CRT)), an alphanumeric input device 912(e.g., a keyboard), a cursor control device 914 (e.g., a mouse), and asignal generation device 920 (e.g., a speaker).

The data storage device 928 may include a machine-readable storagemedium (or more specifically a non-transitory computer-readable storagemedium) 924 on which is stored one or more sets of instructions 926embodying any one or more of the methodologies or functions describedherein. Wherein a non-transitory storage medium refers to a storagemedium other than a carrier wave. The instructions 926 may also reside,completely or at least partially, within the main memory 904 and/orwithin the processing device 902 during execution thereof by thecomputer device 900, the main memory 904 and the processing device 902also constituting computer-readable storage media.

The computer-readable storage medium 924 may also be used to store anadhesive object identification module 950, an image registration moduleand/or a model generation module, which may correspond to similarlynamed components of FIG. 3. The computer readable storage medium 924 mayalso store a software library containing methods that call an adhesiveobject identification module 950, an image registration module and/or amodel generation module. While the computer-readable storage medium 924is shown in an example embodiment to be a single medium, the term“computer-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “computer-readable storage medium” shall also betaken to include any medium that is capable of storing or encoding a setof instructions for execution by the machine and that cause the machineto perform any one or more of the methodologies of the presentinvention. The term “computer-readable storage medium” shall accordinglybe taken to include, but not be limited to, solid-state memories, andoptical and magnetic media.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent upon reading and understanding the above description. Althoughembodiments of the present invention have been described with referenceto specific example embodiments, it will be recognized that theinvention is not limited to the embodiments described, but can bepracticed with modification and alteration within the spirit and scopeof the appended claims. Accordingly, the specification and drawings areto be regarded in an illustrative sense rather than a restrictive sense.The scope of the invention should, therefore, be determined withreference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

What is claimed is:
 1. A method comprising: receiving, by a processingdevice, data from an intraoral scan of a patient responsive toperformance of the intraoral scan being performed by an intraoralscanner, the data comprising a plurality of intraoral images of a dentalsite; determining that at least two intraoral images of the plurality ofintraoral images comprise a representation of at least a portion of anon-rigid object that was affixed to the dental site at a target area,wherein the non-rigid object has known properties comprising at leastone of a known two-dimensional (2D) surface pattern or a known shape,wherein the non-rigid object provides a reference point for imageregistration, wherein the non-rigid object partially deforms based on ashape of the target area, and wherein partial deformation of thenon-rigid object deforms at least one of the known shape or the known 2Dsurface pattern; performing, by the processing device, imageregistration between the at least two intraoral images using thenon-rigid object identified in the at least two intraoral images;generating a three-dimensional (3D) model of the dental site based onthe image registration; determining a surface of a base of the non-rigidobject that has been partially deformed based on the known properties ofthe non-rigid object and the partial deformation of the non-rigidobject; subtracting the representation of the non-rigid object from the3D model based on the known properties of the non-rigid object; andinterpolating a surface of a portion of the dental site where thenon-rigid object was located based on a) data for other portions of thedental site and b) the surface of the base of the non-rigid object. 2.The method of claim 1, wherein the target area comprises an edentulousregion of the dental site, and wherein the non-rigid object comprises anapproximately flat non-rigid adhesive object having a known thickness.3. The method of claim 1, wherein the known properties further comprisea known color of the non-rigid object, the method further comprising:detecting an unknown object in the at least two intraoral images havingthe known color and at least one of the known shape or the known 2Dsurface pattern; and recognizing the unknown object as a representationof the non-rigid object.
 4. The method of claim 1, further comprising:comparing two or more of the plurality of intraoral images to knownproperties of a plurality of different objects to identify the non-rigidobject in the at least two intraoral images.
 5. The method of claim 1,wherein the non-rigid object comprises a plurality of components thateach provide a distinct reference point, the plurality of componentsbeing interconnected by a plurality of links.
 6. The method of claim 1,further comprising: identifying a spatial relationship between aplurality of features of the dental site; determining that a region ofthe dental site comprises one or more missing teeth; determining thatthe region of the dental site lacks features suitable for scanning ofthe dental site; computing an optimal location within the region forplacement for the non-rigid object based on the one or more missingteeth and a field of view of the intraoral scanner; and indicating theoptimal location within the region for placement of the non-rigidobject.
 7. The method of claim 1, wherein a first intraoral image of theat least two intraoral images comprises a first representation of afirst portion of the non-rigid object that is not shown in a secondintraoral image of the at least two intraoral images, wherein the secondintraoral image comprises a second representation of a second portion ofthe non-rigid object that is not shown in the first intraoral image, andwherein the first intraoral image is non-contiguous with the secondintraoral image, the method further comprising: performing, by theprocessing device, pattern recognition to identify the non-rigid objecthaving at least one of the known 2D surface pattern or the known shapein the at least two intraoral images; and extrapolating a shape andorientation of the non-rigid object in at least one of the firstintraoral image or the second intraoral image based on a) at least oneof the known 2D surface pattern or the known shape and b) at least oneof the first representation or the second representation; wherein theimage registration between the first intraoral image and the secondintraoral image is performed using the extrapolated shape andorientation of the non-rigid object in at least one of the firstintraoral image or the second intraoral image.
 8. The method of claim 1,further comprising: receiving an indication that the at least twointraoral images of the plurality of intraoral images comprise therepresentation of at least the portion of the non-rigid object, whereinthe indication is received via a user input.
 9. A non-transitorycomputer readable medium comprising instructions that, when executed bya processing device, cause the processing device to perform operationscomprising: receiving, by the processing device, data from an intraoralscan of a patient responsive to performance of the intraoral scan beingperformed by an intraoral scanner, the data comprising a plurality ofintraoral images of a dental site; determining that at least twointraoral images of the plurality of intraoral images comprise arepresentation of at least a portion of a non-rigid object that wasaffixed to the dental site at a target area, wherein the non-rigidobject has known properties comprising at least one of a knowntwo-dimensional (2D) surface pattern or a known shape, wherein thenon-rigid object provides a reference point for image registration,wherein the non-rigid object partially deforms based on a shape of thetarget area, and wherein partial deformation of the non-rigid objectdeforms at least one of the known shape or the known 2D surface pattern;performing, by the processing device, image registration between the atleast two intraoral images using the non-rigid object identified in theat least two intraoral images; generating a three-dimensional (3D) modelof the dental site based on the image registration; determining asurface of a base of the non-rigid object that has been partiallydeformed based on the known properties of the non-rigid object and thepartial deformation of the non-rigid object; subtracting therepresentation of the non-rigid object from the 3D model based on theknown properties of the non-rigid object; and interpolating a surface ofa portion of the dental site where the non-rigid object was locatedbased on a) data for other portions of the dental site and b) thesurface of the base of the non-rigid object.
 10. The non-transitorycomputer readable medium of claim 9, wherein the target area comprisesan edentulous region of the dental site, and wherein the non-rigidobject comprises an approximately flat non-rigid adhesive object havinga known thickness.
 11. The non-transitory computer readable medium ofclaim 9, wherein the known properties further comprise a known color ofthe non-rigid object, the operations further comprising: detecting anunknown object in the at least two intraoral images having the knowncolor and at least one of the known shape or the known 2D surfacepattern; and recognizing the unknown object as a representation of thenon-rigid object.
 12. The non-transitory computer readable medium ofclaim 9, wherein the data comprises a video comprising the plurality ofintraoral images.
 13. The non-transitory computer readable medium ofclaim 9, wherein the non-rigid object comprises a plurality ofcomponents that each provide a distinct reference point, the pluralityof components being interconnected by a plurality of links.
 14. Thenon-transitory computer readable medium of claim 9, the operationsfurther comprising: identifying a spatial relationship between aplurality of features of the dental site; determining that a region ofthe dental site comprises one or more missing teeth; determining thatthe region of the dental site lacks features suitable for scanning ofthe dental site; computing an optimal location within the region forplacement for the non-rigid object based on the one or more missingteeth and a field of view of the intraoral scanner; and indicating theoptimal location within the region for placement of the non-rigidobject.
 15. The non-transitory computer readable medium of claim 9,wherein a first intraoral image of the at least two intraoral imagescomprises a first representation of a first portion of the non-rigidobject that is not shown in a second intraoral image of the at least twointraoral images, wherein the second intraoral image comprises a secondrepresentation of a second portion of the non-rigid object that is notshown in the first intraoral image, and wherein the first intraoralimage is non-contiguous with the second intraoral image, the operationsfurther comprising: performing, by the processing device, patternrecognition to identify the non-rigid object having at least one of theknown 2D surface pattern or the known shape in the at least twointraoral images; and extrapolating a shape and orientation of thenon-rigid object in at least one of the first intraoral image or thesecond intraoral image based on a) at least one of the known 2D surfacepattern or the known shape and b) at least one of the firstrepresentation or the second representation; wherein the imageregistration between the first intraoral image and the second intraoralimage is performed using the extrapolated shape and orientation of thenon-rigid object in at least one of the first intraoral image or thesecond intraoral image.
 16. The non-transitory computer readable mediumof claim 9, the operations further comprising: receiving an indicationthat the at least two intraoral images of the plurality of intraoralimages comprise the representation of at least the portion of thenon-rigid object, wherein the indication is received via a user input.17. A system comprising: an intraoral scanner to generate an intraoralscan of a patient; and a computing device operatively coupled to theintraoral scanner, the computing device to: receive data from theintraoral scan of the patient, the data comprising a plurality ofintraoral images of a dental site; determine that at least two intraoralimages of the plurality of intraoral images comprise a representation ofat least a portion of a non-rigid object that was affixed to the dentalsite at a target area, wherein the non-rigid object has known propertiescomprising at least one of a known two-dimensional (2D) surface patternor a known shape, wherein the non-rigid object provides a referencepoint for image registration, wherein the non-rigid object partiallydeforms based on a shape of the target area, and wherein partialdeformation of the non-rigid object deforms at least one of the knownshape or the known 2D surface pattern; perform image registrationbetween the at least two intraoral images using the non-rigid objectidentified in the at least two intraoral images; generate athree-dimensional (3D) model of the dental site based on the imageregistration; determine a surface of a base of the non-rigid object thathas been partially deformed based on the known properties of thenon-rigid object and the partial deformation of the non-rigid object;subtract the representation of the non-rigid object from the 3D modelbased on the known properties of the non-rigid object; and interpolate asurface of a portion of the dental site where the non-rigid object waslocated based on a) data for other portions of the dental site and b)the surface of the base of the non-rigid object.
 18. The system of claim17, further comprising a data store operatively coupled to at least oneof the intraoral scanner or the computing device, the data store tostore the data from the intraoral scan of the patient and the knownproperties of the non-rigid object.
 19. The system of claim 17, whereinthe known properties further comprise a known color of the non-rigidobject, and wherein the computing device is further to: detect anunknown object in the at least two intraoral images having the knowncolor and at least one of the known shape or the known 2D surfacepattern; and recognize the unknown object as a representation of thenon-rigid object.
 20. The system of claim 17, wherein a first intraoralimage of the at least two intraoral images comprises a firstrepresentation of a first portion of the non-rigid object that is notshown in a second intraoral image of the at least two intraoral images,wherein the second intraoral image comprises a second representation ofa second portion of the non-rigid object that is not shown in the firstintraoral image, and wherein the first intraoral image is non-contiguouswith the second intraoral image, the computing device further to:perform pattern recognition to identify the non-rigid object having atleast one of the known 2D surface pattern or the known shape in the atleast two intraoral images; and extrapolate a shape and orientation ofthe non-rigid object in at least one of the first intraoral image or thesecond intraoral image based on a) at least one of the known 2D surfacepattern or the known shape and b) at least one of the firstrepresentation or the second representation; wherein the imageregistration between the first intraoral image and the second intraoralimage is performed using the extrapolated shape and orientation of thenon-rigid object in at least one of the first intraoral image or thesecond intraoral image.